Journal of Educational Evaluation for Health Professions

Research articles

Possibility of independent use of the yes/no Angoff and Hofstee methods for the standard setting of the Korean Medical Licensing Examination written test: a descriptive study: Do-Hwan Kim, Ye Ji Kang, Hoon-Ki Park; J Educ Eval Health Prof. 2022;19:33. Published online December 12, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.33

1,635 View
113 Download
2 Web of Science
2 Crossref

Purpose
This study aims to apply the yes/no Angoff and Hofstee methods to actual Korean Medical Licensing Examination (KMLE) 2022 written examination data to estimate cut scores for the written KMLE.
Methods
Fourteen panelists gathered to derive the cut score of the 86th KMLE written examination data using the yes/no Angoff method. The panel reviewed the items individually before the meeting and shared their respective understanding of the minimum-competency physician. The standard setting process was conducted in 5 rounds over a total of 800 minutes. In addition, 2 rounds of the Hofstee method were conducted before starting the standard setting process and after the second round of yes/no Angoff.
Results
For yes/no Angoff, as each round progressed, the panel’s opinion gradually converged to a cut score of 198 points, and the final passing rate was 95.1%. The Hofstee cut score was 208 points out of a maximum 320 with a passing rate of 92.1% at the first round. It scored 204 points with a passing rate of 93.3% in the second round.
Conclusion
The difference between the cut scores obtained through yes/no Angoff and Hofstee methods did not exceed 2% points, and they were within the range of cut scores from previous studies. In both methods, the difference between the panelists decreased as rounds were repeated. Overall, our findings suggest the acceptability of cut scores and the possibility of independent use of both methods.

Citations

Citations to this article as recorded by

Issues in the 3rd year of the COVID-19 pandemic, including computer-based testing, study design, ChatGPT, journal metrics, and appreciation to reviewers
Sun Huh
Journal of Educational Evaluation for Health Professions.2023; 20: 5. CrossRef
Presidential address: improving item validity and adopting computer-based testing, clinical skills assessments, artificial intelligence, and virtual reality in health professions licensing examinations in Korea
Hyunjoo Pai
Journal of Educational Evaluation for Health Professions.2023; 20: 8. CrossRef

Is it possible to introduce an interview to the Korean Medical Licensing Examination to assess professional attributes?: a survey-based observational study: Seung-Joo Na, HyeRin Roh, Kyung Hee Chun, Kyung Hye Park, Do-Hwan Kim; J Educ Eval Health Prof. 2022;19:10. Published online May 10, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.10

3,032 View
285 Download

Abstract PDF Supplementary Material: Purpose
This study aimsed to gather opinions from medical educators on the possibility of introducing an interview to the Korean Medical Licensing Examination (KMLE) to assess professional attributes. Specifically following topics were dealt with: the appropriate timing and tool to assess unprofessional conduct; ; the possiblity of prevention of unprofessional conduct by introducing an interview to the KMLE; and the possibility of implementation of an interview to the KMLE.
Methods
A cross-sectional study approach based on a survey questionnaire was adopted. We analyzed 104 pieces of news about doctors’ unprofessional conduct to determine the deficient professional attributes. We derived 24 items of unprofessional conduct and developed the questionnaire and surveyed 250 members of the Korean Society of Medical Education 2 times. Descriptive statistics, cross-tabulation analysis, and Fisher’s exact test were applied to the responses. The answers to the open-ended questions were analyzed using conventional content analysis.
Results
In the survey, 49 members (19.6%) responded. Out of 49, 24 (49.5%) responded in the 2nd survey. To assess unprofessional conduct, there was no dominant timing among basic medical education (BME), KMLE, and continuing professional development (CPD). There was no overwhelming assessment tool among written examination, objective structured clinical examination, practice observation, and interview. Response rates of “impossible” (49.0%) and “possible” (42.9%) suggested an interview of the KMLE prevented unprofessional conduct. In terms of implementation, “impossible” (50.0%) was selected more often than “possible” (33.3%).
Conclusion
Professional attributes should be assessed by various tools over the period from BME to CPD. Hence, it may be impossible to introduce an interview to assess professional attributes to the KMLE, and a system is needed such as self-regulation by the professional body rather than licensing examination.

Changes in academic performance in the online, integrated system-based curriculum implemented due to the COVID-19 pandemic in a medical school in Korea: Do-Hwan Kim, Hyo Jeong Lee, Yanyan Lin, Ye Ji Kang; J Educ Eval Health Prof. 2021;18:24. Published online September 23, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.24

5,809 View
310 Download
11 Web of Science
11 Crossref

Abstract PDF Supplementary Material

Purpose
This study examined how students’ academic performance changed after undergoing a transition to online learning during the coronavirus disease 2019 (COVID-19) pandemic, based on the test results of 16 integrated courses conducted in 3 semesters at Hanyang University College of Medicine in Korea.
Methods
For the 16 required courses that formed an integrated system-based curriculum running for 3 semesters, the major examinations’ raw scores were collected for each student. Percent-correct scores were used in the subsequent analysis. We used the t-test to compare grades between 2019 and 2020, and the Cohen D was calculated as a measure of effect size. The correlation of scores between courses was calculated using Pearson correlation coefficients.
Results
There was a significant decrease in scores in 2020 for 10 courses (62.5%). While most of the integrated system-based curriculum test scores showed strong correlations, with coefficients of 0.6 or higher in both 2019 and 2020, the correlation coefficients were generally higher in 2020. When students were divided into low, middle, and high achievement groups, low-achieving students consistently showed declining test scores in all 3 semesters.
Conclusion
Our findings suggest that the transition to online classes due to COVID-19 has led to an overall decline in academic performance. This overall decline, which may occur when the curriculum is centered on recorded lectures, needs to be addressed. Further, medical schools need to consider establishing a support system for the academic development of low-achieving students.

Citations

Citations to this article as recorded by

Study delay during emergency remote teaching among students at Dutch universities: the role of students’ education satisfaction and academic wellbeing
Manja Vollmann, Renée A. Scheepers, Femke Hilverda
European Journal of Psychology of Education.2024;[Epub] CrossRef
Impact of the Covid-19 pandemic and ensuing online teaching on pre-clinical medical education
Houman Goudarzi, Masahiro Onozawa, Makoto Takahashi
BMC Medical Education.2024;[Epub] CrossRef
Navigating the shift to online learning: student experiences of inclusivity, efficiency, and study efforts in Chile
Grace Melo, Diego Monteza, Sandra Acosta
Cogent Education.2024;[Epub] CrossRef
The nationwide impact of COVID-19 on life support courses. A retrospective evaluation by Resuscitation Council UK
C.J. Thorne, P.K. Kimani, S. Hampshire, I. Hamilton-Bower, S. Begum-Ali, A. Benson-Clarke, K. Couper, J. Yeung, A. Lockey, G.D. Perkins, J. Soar
Resuscitation Plus.2023; 13: 100366. CrossRef
COVID-19 and Living Situation: Effects on University of Michigan Students During the 2020-2021 Academic Year
Barbara Tan, Alyssa Cadez-Martin, Sarah Fox Fox, Samir Gadepalli
University of Michigan Undergraduate Research Journal.2023;[Epub] CrossRef
How successful the online assessment techniques in distance learning have been, in contributing to academic achievements of management undergraduates?
Hiruni Thathsarani, Dinushika Kaushalya Ariyananda, Chalani Jayakody, Kerthiga Manoharan, A.A.S.N Munasinghe, Nilmini Rathnayake
Education and Information Technologies.2023; 28(11): 14091. CrossRef
Academic performance and mental health among Chinese middle and high school students after the lifting of COVID-19 restrictions
Hong-Jun Song, Yun-Fei Mu, Cong Wang, Jia Cai, Zhong-Yue Deng, Ai-Ping Deng, Xue-Hua Huang, Xian-Dong Meng, Lan Zhang, Yi Huang, Wei Zhang, Wen-Wu Shen, Jin Chen, Bo Liu, Ru Gao, Jun-Shu Zhao, Mao-Sheng Ran
Frontiers in Psychiatry.2023;[Epub] CrossRef
HOW DID ONLINE LEARNING AFFECT PHYSIOTHERAPY STUDENTS DURING THE COVID-19 PANDEMIC?
Muhammet Fatih BULUT, Aybüke Cansu KALKAN, Arzu GENÇ, Seher ÖZYÜREK
SDÜ Tıp Fakültesi Dergisi.2023; 30(4): 652. CrossRef
The impact of asynchronous online anatomy teaching and smaller learning groups in the anatomy laboratory on medical students’ performance during the Covid‐19 pandemic
Ming‐Fong Chang, Meng‐Lin Liao, June‐Horng Lue, Chi‐Chuan Yeh
Anatomical Sciences Education.2022; 15(3): 476. CrossRef
Educational impact of an active learning session with 6-lead mobile electrocardiography on medical students’ knowledge of cardiovascular physiology during the COVID-19 pandemic in the United States: a survey-based observational study
Alexandra Camille Greb, Emma Altieri, Irene Masini, Emily Helena Frisch, Milton Leon Greenberg
Journal of Educational Evaluation for Health Professions.2022; 19: 12. CrossRef
Coronavirus Disease 2019 Cases at Universities and Colleges in Seoul Metropolitan Area
Young June Choe, Yun-Kyung Kim
Journal of Korean Medical Science.2021;[Epub] CrossRef

First
Prev
Page of 1
Next
Last

Search